In order to objectively and accurately evaluate the image fusion algorithms, an evaluation algorithm based on TV-L1 (Total Variation regularization) structure and texture decomposition was proposed. According to the studies on human visual system, human's perception to image quality mainly comes from the underlying visual features of image, and structure features and texture features are the most important features of underlying visual feature of image. However, the existed image fusion quality evaluation algorithms ignore this fact and lead to inaccurate evaluation. To address this problem, a pair of source images and their corresponding fusion results were individually decomposed into structure and texture images with a two-level TV-L1 decomposition. Then, According to the difference of image features between the structure and texture images, the similarity evaluation was carried out from the decomposed structure image and the texture image respectively, and the final evaluation score was obtained by integrating the scores at all levels. Based on the dataset with 30 images and 8 mainstream fusion algorithms, compared with the 11 existing objective evaluation indexes, the Borda counting method and Kendall coefficient were employed to verify the consistency of the proposed evaluation algorithm. Moreover, the consistency between the proposed objective evaluation index and the subjective evaluation is verified on the subjective evaluation image set.